A chip multi-processor (CMP) is a single integrated circuit (chip) having several tiles and one or more memory controllers. Each tile includes a core, one or more caches, and a router. Some CMPs have 256 or more tiles. The core typically contains an arithmetic logic unit or other circuits that can respond to machine instructions to perform logical, arithmetic, or other computer operations. The cache may include a private cache and lower-level caches. The last level cache can be private or shared through aggregation. The router handles communications. Data (including machine instructions and information such as numbers and text on which the instructions operate) can be retrieved from the cache much more quickly than from system memory. Initially, data resides in some form of non-volatile storage such as a hard disk. When an application is to be run, data pertaining to that application is transferred from storage into system memory. From there, items of data are copied into cache as the items are needed. Ideally, those items that are used most often are the ones in the cache. An application “sees” an item as having an address that is referred to as a virtual address. A page table in memory correlates virtual addresses as seen by an application with physical memory addresses, including auxiliary bits to aid the correlation. Portions of the page table may be copied into a translation lookaside buffer (TLB) for rapid access. Directory hardware keeps track of the physical addresses of items that have been copied into cache. The directory hardware can be distributed amongst the tiles to reduce overhead. When an application running in a tile requests an item having a certain virtual address, the TLB is consulted to find the corresponding physical address. The tile checks its local private cache (the private cache may have one or several levels), and if the item is not there or if the private cache does not have appropriate permissions, then the directory hardware is consulted to determine whether the requested item has been copied into cache. If so, the directory hardware identifies that tile having the cache which contains the copy of the requested item. Then the physical address is used to look up the item in that cache.